Online Learning with Low Rank Experts

We consider the problem of prediction with expert advice when the losses of the experts have low-dimensional structure: they are restricted to an unknown d-dimensional subspace. We devise algorithms with regret bounds that are independent of the number of experts and depend only on the rank d. For the stochastic model we show a tight bound of Θp ? dT q, and extend it to a setting of an approximate d subspace. For the adversarial model we show an upper bound of Opd ? T q and a lower bound of Ωp ? dT q.